Automated Ligand Design in Simulated Molecular Docking

The drug discovery process broadly follows the sequence of high-throughput screening, optimisation, synthesis, testing, and finally, clinical trials. We investigate methods for accelerating this process with machine learning algorithms that can automatically design novel ligands for biological targets. Recent work has demonstrated the viability of deep reinforcement learning, generative adversarial networks and auto-encoders. Here, we extend state-of-the-art deep reinforcement learning molecular modification algorithms and, through the integration of molecular docking simulations, apply them to automatically design novel antagonists for the adenosine triphosphate binding site of Plasmodium falciparum phosphatidylinositol 4-kinase, an enzyme essential to the malaria parasite’s development within an infected host. We demonstrated that such an algorithm was capable of designing novel molecular graphs with better DSs than the best DSs in a set of reference molecules. There reference set here was a set of 1,011 structural analogues of napthyridine, imidazopyridazine, and aminopyradine

Neuro-Evolution for Multi-Agent Policy Transfer in RoboCup Keep-Away

An objective of transfer learning is to improve and speedup learning on target tasks after training on a different, but related source tasks. This research is a study of comparative Neuro-Evolution (NE) methods for transferring evolved multi-agent policies (behaviors) between multi-agent tasks of varying complexity. The efficacy of five variants of two NE methods are compared for multi-agent policy transfer. The NE method variants include using the original versions (search directed by a fitness function), behavioural and genotypic diversity based search to replace objective based search (fitness functions) as well as hybrid objective and diversity (behavioral and genotypic) maintenance based search approaches. The goal of testing these variants to direct policy search is to ascertain an appropriate method for boosting the task performance of transferred multi-agent behaviours. Results indicate that an indirect encoding NE method using hybridized objective based search and behavioral diversity maintenance yields significantly improved task performance for policy transfer between multi-agent tasks of increasing complexity. Comparatively, NE methods not using behavioral diversity maintenance to direct policy search performed relatively poorly in terms of efficiency (evolution times) and quality of solutions in target tasks

Automated Pattern Identification and Classification: Anomaly Detection Case Study

In this study, the efficacy of the Automated Pattern Identification and Classification (APIC) Machine Learning (ML) pipeline method was evaluated as an Anomaly Intrusion Detection (AID) system to determine if using an ML-pipeline method could reduce false positive rates compared to similar methods using the same data set

Multi-Agent Behavior-Based Policy Transfer

A key objective of transfer learning is to improve and speedup learning on a target task after training on a different, but related, source task. This study presents a neuro-evolution method that transfers evolved policies within multi-agent tasks of varying degrees of complexity. The method incorporates behavioral diversity (novelty) search as a means to boost the task performance of transferred policies (multi-agent behaviors). Results indicate that transferred evolved multi-agent behaviors are significantly improved in more complex tasks when adapted using behavioral diversity. Comparatively, behaviors that do not use behavioral diversity to further adapt transferred behaviors, perform relatively poorly in terms of adaptation times and quality of solutions in target tasks. Also, in support of previous work, both policy transfer methods (with and without behavioral diversity adaptation), out-perform behaviors evolved in target tasks without transfer learning

Towards Run-time Efficient Hierarchical Reinforcement Learning

This paper investigates a novel method combining Scalable Evolution Strategies (S-ES) and Hierarchical Reinforcement Learning (HRL). S-ES, named for its excellent scalability, was popularised with demonstrated performance comparable to state-of-the-art policy gradient methods. However, S-ES has not been tested in conjunction with HRL methods, which empower temporal abstraction thus allowing agents to tackle more challenging problems. We introduce a novel method merging S-ES and HRL, which creates a highly scalable and efficient (compute time) algorithm. We demonstrate that the proposed method benefits from S-ES’s scalability and indifference to delayed rewards. This results in our main contribution: significantly higher learning speed and competitive performance compared to gradient-based HRL methods, across a range of tasks

Deriving Minimal Sensory Configurations for Evolved Cooperative Robot Teams

This paper presents a study on the impact of different robot sensory configurations (morphologies) in simulated robot teams that must accomplish a collective (cooperative) behavior task. The study’s objective was to investigate if effective collective behaviors could be efficiently evolved given minimal morphological complexity of individual robots in an homogenous team. A range of sensory configurations are tested in company with evolved controllers for a collective construction task. Results indicate that a minimal sensory configuration yields the highest task performance, and increasing the complexity of the sensory configuration does not yield an increased task performance

Hybridizing Novelty Search for Transfer Learning

This study investigates the impact of genotypic and behavioral diversity maintenance methods on controller evolution in multi-robot (RoboCup keep-away soccer) tasks. The focus is to examine the impact of these methods on the transfer learning of behaviors, first evolved in a source task before being transferred for further evolution in different but related target tasks. The goal is to ascertain an appropriate controller design (NE: NeuroEvolution) method for facilitating improved effectiveness given policy transfer between source and target tasks. Effectiveness is defined as the average task performance of transferred behaviors. The study comparatively tests and evaluates the efficacy of coupling policy transfer with several NE variants. Results indicate a hybrid of behavioral diversity maintenance and objective-based search yields significantly improved effectiveness for evolved behaviors across increasingly complex target tasks. Results also highlight the efficacy of coupling policy transfer with the hybrid of behavioral diversity maintenance and objective based search in order to address bootstrapping and deception problems endemic to complex tasks

The Benefits of Adaptive Behavior and Morphology for Cooperation in Robot Teams

This is a study on the role of morphological (sensor configuration) and behavioral (control system) adaptation in simulated robot teams that must accomplish cooperative tasks. The research objective was to elucidate the necessary features and computational mechanics of a method that automates the behavior-morphology design of robot teams that must accomplish cooperative tasks (tasks that cannot be optimally solved by individual robots). Results indicate that automating behavior-morphology design is beneficial as task complexity increases, compared to evolving behaviors in fixed morphology teams. However, increased task complexity does not necessarily equate to the evolution of increased morphological complexity in teams